Systematic Reviews of Genetic Association Studies

Gurdeep S. Sagoo and colleagues describe key components of the methodology for undertaking systematic reviews and meta-analyses of genetic association studies

Rethinking the assessment of risk of bias due to selective reporting:a cross-sectional study

BACKGROUND: Selective reporting is included as a core domain of Cochrane’s tool for assessing risk of bias in randomised trials. There has been no evaluation of review authors’ use of this domain. We aimed to evaluate assessments of selective reporting in a cross-section of Cochrane reviews and to outline areas for improvement. METHODS: We obtained data on selective reporting judgements for 8434 studies included in 586 Cochrane reviews published from issue 1–8, 2015. One author classified the reasons for judgements of high risk of selective reporting bias. We randomly selected 100 reviews with at least one trial rated at high risk of outcome non-reporting bias (non-/partial reporting of an outcome on the basis of its results). One author recorded whether the authors of these reviews incorporated the selective reporting assessment when interpreting results. RESULTS: Of the 8434 studies, 1055 (13 %) were rated at high risk of bias on the selective reporting domain. The most common reason was concern about outcome non-reporting bias. Few studies were rated at high risk because of concerns about bias in selection of the reported result (e.g. reporting of only a subset of measurements, analysis methods or subsets of the data that were pre-specified). Review authors often specified in the risk of bias tables the study outcomes that were not reported (84 % of studies) but less frequently specified the outcomes that were partially reported (61 % of studies). At least one study was rated at high risk of outcome non-reporting bias in 31 % of reviews. In the random sample of these reviews, only 30 % incorporated this information when interpreting results, by acknowledging that the synthesis of an outcome was missing data that were not/partially reported. CONCLUSIONS: Our audit of user practice in Cochrane reviews suggests that the assessment of selective reporting in the current risk of bias tool does not work well. It is not always clear which outcomes were selectively reported or what the corresponding risk of bias is in the synthesis with missing outcome data. New tools that will make it easier for reviewers to convey this information are being developed. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s13643-016-0289-2) contains supplementary material, which is available to authorized users

Data mining in clinical trial text: transformers for classification and question answering tasks

This research on data extraction methods applies recent advances in natural language processing to evidence synthesis based on medical texts. Texts of interest include abstracts of clinical trials in English and in multilingual contexts. The main focus is on information characterized via the Population, Intervention, Comparator, and Outcome (PICO) framework, but data extraction is not limited to these fields. Recent neural network architectures based on transformers show capacities for transfer learning and increased performance on downstream natural language processing tasks such as universal reading comprehension, brought forward by this architecture’s use of contextualized word embeddings and self-attention mechanisms. This paper contributes to solving problems related to ambiguity in PICO sentence prediction tasks, as well as highlighting how annotations for training named entity recognition systems are used to train a high-performing, but nevertheless flexible architecture for question answering in systematic review automation. Additionally, it demonstrates how the problem of insufficient amounts of training annotations for PICO entity extraction is tackled by augmentation. All models in this paper were created with the aim to support systematic review (semi)automation. They achieve high F1 scores, and demonstrate the feasibility of applying transformer-based classification methods to support data mining in the biomedical literature

The range of peripapillary retinal nerve fibre layer and optic disc parameters in children aged up to but not including 18 years of age, as measured by optical coherence tomography:protocol for a systematic review

BACKGROUND: The parameters of the optic disc and peripapillary retinal nerve fibre layer (pRNFL) in children may vary with disease processes that contribute to visual impairment and blindness and so could be useful as an objective measure in at-risk children. There is no standardised reference for the normal parameters of the optic disc and pRNFL in children; however, there are a large number of small individual studies that have been undertaken to look at these measures. METHODS: A systematic review of current literature on the range of pRNFL and optic disc parameters in children aged less than 18 years will be performed. Studies will be considered for review if they report numerical data on optic disc and pRNFL parameters, measured using optical coherence tomography. Outcome measures will include mean pRNFL thickness and cup-disc ratio. The bibliographic databases Medline, CINAHL, EMBASE, Scopus and Web of Science will be systematically searched from 1991. Screening of search results will be conducted by two authors working independently, as will extraction of primary and secondary outcome data. Ten per cent of all other data extraction will be checked by a second author. Results will be compiled and presented in evidence tables. Where possible and appropriate, study-specific estimates will be combined to obtain an overall summary estimate of pRNFL thickness and cup-disc ratio across studies and results will be presented by age of population. Subgroup analyses will be undertaken for children of different ethnicities. DISCUSSION: This review aims to provide an overview of the parameters of the optic disc and pRNFL in children of different ages in order to identify gaps in knowledge and to improve understanding of what might be considered within/outside the range of normality. The findings will be presented in peer-reviewed journals and will be presented at conferences. SYSTEMATIC REVIEW REGISTRATION: PROSPERO CRD42016033068 ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s13643-016-0247-z) contains supplementary material, which is available to authorized users

Graphical augmentations to the funnel plot assess the impact of additional evidence on a meta-analysis

AbstractObjectiveWe aim to illustrate the potential impact of a new study on a meta-analysis, which gives an indication of the robustness of the meta-analysis.Study Design and SettingA number of augmentations are proposed to one of the most widely used of graphical displays, the funnel plot. Namely, 1) statistical significance contours, which define regions of the funnel plot in which a new study would have to be located to change the statistical significance of the meta-analysis; and 2) heterogeneity contours, which show how a new study would affect the extent of heterogeneity in a given meta-analysis. Several other features are also described, and the use of multiple features simultaneously is considered.ResultsThe statistical significance contours suggest that one additional study, no matter how large, may have a very limited impact on the statistical significance of a meta-analysis. The heterogeneity contours illustrate that one outlying study can increase the level of heterogeneity dramatically.ConclusionThe additional features of the funnel plot have applications including 1) informing sample size calculations for the design of future studies eligible for inclusion in the meta-analysis; and 2) informing the updating prioritization of a portfolio of meta-analyses such as those prepared by the Cochrane Collaboration

Machine learning to assist risk of bias assessments in systematic reviews

Background: Risk-of-bias assessments are now a standard component of systematic reviews. At present, reviewers need to manually identify relevant parts of research articles for a set of methodological elements that affect the risk of bias, in order to make a risk-of-bias judgement for each of these elements. We investigate the use of text mining methods to automate risk-of-bias assessments in systematic reviews. We aim to identify relevant sentences within the text of included articles, to rank articles by risk of bias and to reduce the number of risk-of-bias assessments that the reviewers need to perform by hand. Methods: We use supervised machine learning to train two types of models, for each of the three risk-of-bias properties of sequence generation, allocation concealment and blinding. The first model predicts whether a sentence in a research article contains relevant information. The second model predicts a risk-of-bias value for each research article. We use logistic regression, where each independent variable is the frequency of a word in a sentence or article, respectively. Results: We found that sentences can be successfully ranked by relevance with area under the receiver operating characteristic (ROC) curve (AUC) > 0.98. Articles can be ranked by risk of bias with AUC > 0.72. We estimate that more than 33% of articles can be assessed by just one reviewer, where two reviewers are normally required. Conclusions: We show that text mining can be used to assist risk-of-bias assessments